1. Field of the Invention
The present invention relates to a method of positioning a hand of a user and an indicating point like a fingertip as well as a method of determining the hand gesture of the user, used in a human-computer interaction system.
2. Description of the Related Art
Video has been applied to various fields of the society with the rapid development of computer techniques and image processing techniques. Analysis and processing of the video has been become a hot research topic in the current image processing field. A key processing technique is carrying out detection with regard to moving objects or targets in the video, and the detected results are usually used for higher level analysis and processing such as target tracking, classification and others. In various meetings, it is helpful for carrying out a presentation to adopt a projector as an assistant tool. However, during the presentation, a presenter sometimes needs to point out contents projected on a projection area so that audience may combine what they heard with the displayed contents, or needs to carry out interactive operations such as turning a slide, drawing a line, annotating a text segment, etc. In general, these kinds of interactive operations are achieved based on handling a mouse of a computer by a presenter; in other words, the presenter needs to walk back and forth between the computer and a projection screen. It is apparent that this kind of handling method is very inconvenient for the presenter. As a result, it is desirable to provide a human-computer interaction (HCI) system by which it is possible to directly carry out interaction with contents projected on a projection area by using a hand of a user himself or an object like a rod in the hand, or simply speaking, by which it is possible to use a finger of a user to replace some functions of a mouse.
In the below cited reference No. 1, a vision-based gesture interface system, which employs an arbitrary quadrangle-shaped panel and a pointer tip like a fingertip as an intuitive input device, is disclosed.
In the below cited reference No. 2, a drawing, writing, or pointing device for use with data presentation, computer-supported work, or other interactive use of a computer is disclosed.
In the below cited references No. 3 and 4, a technique of background subtraction is adopted for extracting foreground, and a skin color detector is used for removing a non-hand object such as a collar, etc., in a foreground image so as to realize hand detection.
In the following cited reference No. 5, possible arm areas are obtained by carrying out skin color detection, and then detection of the real arm area is realized by supposing that the fastest moving center of mass is the arm that is currently behaving.
Furthermore, in the following cited reference No. 6, a method of detecting fingertip shape by seeking two local features is disclosed. One local feature is that the internal portion of a fingertip is usually surrounded by a circle formed by filled pixels i.e. foreground pixels. Another local feature is that the internal portion of a square, outside ball of a finger is usually surrounded by a long chain formed by non-filled pixels and a short chain formed by filled pixels.
It is apparent that real time processing is an important evaluation index of a human-computer interaction system. In order to achieve the real time processing, in many methods, a feature that may be easily detected (for example, a quadrangle-shaped panel or a marked object) is introduced, detection of skin color is adopted in a whole or partial video image, or movement information is used for accelerating the speed of hand detection. However, the introduced feature that may be easily detected is not convenient enough for applications, and the skin color and the movement information may not be obtained under some conditions of use. As a result, it is worthwhile to seek a rapid method of detecting and positioning a hand and a finger. At the same time, as for a detection problem where a finger shape feature is not obvious due to the bad viewing angle of a single-lens video camera, the above-mentioned conventional methods cannot provide an effective way to solve the detection problem.
Cited Reference No. 1: U.S. Pat. No. 6,594,616 B2
Cited Reference No. 2: US Patent Application Publication NO. 2003/0178493 A1
Cited Reference No. 3: Shahzad Malik (smalik@cs.toronto.edu), “Real-Time Hand Tracking and Finger Tracking for Interaction”, CSC2503F Project Report
Cited Reference No. 4: Son DoLenh, “Multi-Finger Interactions with Paper Augmented Tabletops”
Cited Reference No. 5: “A Virtual 3D Blackboard: 3D Finger Tracking Using a Single Camera”, University of Central Florida
Cited Reference No. 6: “Bare-Hand Human-Computer Interaction”, Proceedings of the ACM Workshop on Perceptive User Interface 2001